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roecom^  Ljfe  Cycle  RAM  Activities 


Pre-Milestone  A/Concept  Refinement 

•  ICD  Requirements  Feasibility  Study 

•  Purchase  Description  Development 

•  Scope  of  Work  Development  (Sections  C  and  E) 

•  Whole  Systems  Trade  Studies  (WSTS) 

•  Test  and  Evaluation  Strategy  (TES)  Development 

•  Participate  in  SRR 

•  Analysis  Of  Alternatives  (AoA) 

•  Similar  System  Analysis 

•  Trades/Gap  Analysis 

•  T&E  Strategy 

•  Failure  Definition  Scoring  Criteria  (FDSC) 

•  Reliability  Growth  Plan 

•  Source  Selection 
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rdecom^  Life  cycle  RAM  Activities 


Milestone  A-B/Technology  Development 

•  CDD  Requirements  Feasibility  Study 

•  Purchase  Description  Development 

•  Scope  of  Work  Development  (sections  C  and  E) 

•  TEMP  Development 

•  RAM-D  Report  Development 

•  Participate  in  Design  Reviews  (SRR,  ISR,  PDR,  CDR,  TRR) 

•  Analysis  Of  Alternatives  (AoA) 

•  Failure  Modes  Effects  and  Criticality  Analysis  (FMECA) 

•  Failure  Definition  Scoring  Criteria  (FDSC) 

•  Risk  Assessment 

•  T&E  Strategy 

•  Reliability  Growth  Tracking 

•  Source  Selection 

•  RAM  Test  Execution 

-  Test  Incident  Report  (TIR)  Review 

-  Chair  Scoring  Conference 

-  Corrective  Action  Review  Board 

-  Assessment  Conferences 

-  Analysis 
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roecom^  Ljfe  Cycle  RAM  Activities 


Milestone  B-C / 

Engineering  &  Manufacturing  Development 

*  DT&E  Results 

*  Participate  in  Design  Reviews  (SRR,  IBR,  PDR,  CDR,  TRR) 

*  Purchase  Description  Development 

*  Scope  of  Work  Development  (Sections  C  and  E) 

*  TEMP  Development 

*  Failure  Definition  Scoring  Criteria 

*  RAM-D  Report  Development 

*  CPD  Development 

*  T&E  Strategy 

*  Reliability  Growth  Tracking 

*  Source  Selection 

*  Developmental  RAM  Test  Execution 

-  Test  Incident  Report  (TIR)  Review 

-  Chair  Scoring  conference 

-  Corrective  Action  Review  Board 

-  Assessment  Conferences 

-  Analysis 
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rdfmsom^  Life  Cycle  RAM  Activities 


Post-  Milestone  C/Production  &  Deployment 

•  LRIP/  Production  Contract 

•  PQT/PVT  Results 

•  Source  Selection 

•  Reliability  Growth  Tracking 

•  Operational  RAM  Test  Execution 

-  Test  Incident  Report  (TIR)  Review 

-  Scoring  Conference 

-  Corrective  Action  Review  Board 

-  Assessment  Conferences 

-  Analysis 

•  Field  and  System  Evaluation 

-  Test  Incident  Report  (TIR)  Review 

-  Initiate  Cost  Reductions  (OSCR  Project  Support) 

-  AMSAA  /  OSMIS  / 1  LAP  Data  Analysis 

-  ECP  Assessments 
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RDFttoM^  Life  Cycle  RAM  Activities 


Systems  Supported: 


JLTV 

PIM 

STRYKER 

RCV 

FMTV 

M915A5 


HEMTT 

MRAP 

HMMWV 

HETS 

GCV 

PLS 


OTHERS 
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RDECOM 


RAM  -  Requirements  Development 


•  Review  User’s  draft  CDD  /  FDSC  /  OMS-MP 

•  Perform  Materiel  Developer’s  Analysis  using  predecessor 
or  similar  system 

•  Utilize  previous  test  data,  VDLS,  OSMIS,  Army  ILAP,  RAM-D 
summaries 

Comparison  of  failure  definition 
Comparison  of  mission  profile  and  weight 
Perform  state-of-art  analysis 

•  Determine  feasibility  of  User’s  requirements  for 
Reliability,  Maintainability,  and  Availability 

•  Mean  time  between  failure  (MTBF,  MMBSA,  MMBEFF,  MRBF,  etc...) 

Maintenance  ratio  (MR),  Mean  time  to  repair  (MTTR,  MaxTTR, 
etc...) 

Material  Availability  (Am),  Operational  Availability  (Ao),  Inherent 
Availability  (Ai),  etc... 

•  Provide  comments  and  reasoning  for  all  recommendations 
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RAM  -  Requirements  Pevelopment^^^ 


•  Establish  RAM  scope  of  test 

•  Negotiate  RAM  test  scope  (Number  of  vehicles,  number  of 
miles,  number  of  rounds  fired) 

To  effectively  prove  out  R&M  requirements 

•  while  minimizing  cost 

•  and  minimizing  the  schedule  impact  on  the  program 

•  Provide  RAM  input  to  Spec  and  SOW 

•  Convert  user’s  operational  requirements  into  technical 
requirements  for  SPEC 

•  Provide  comments  and  reasoning  for  all  recommendations 

•  Develop  contract  language  for  SOW 

Using  the  latest  DOD  guide  and  ASA(ALT)  guideline  of  best  reliability  practice 

•  Coordinate  RAM  contract  languages  among  RAM  IPT 

•  Provide  RAM  language  to  various  technical  documents 

•  TEMP 


•  AS 
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RAM  Involvement  during 
Source  Selection 


^  ss  \\  Y 


•  TARDEC  RAM  Engineering  provides  support  during  source 
selection  preparations. 

•  TARDEC  RAM  Engineering  also  support  bid  sample  testing 
during  source  selection. 

•  During  the  conduct  of  a  Source  Selection  Evaluation  Board 
(SSEB),  TARDEC  RAM  Engineers  serve  as  subject  matter 
experts. 

•  TARDEC  RAM  Engineers  participate  in  face  to  face 
discussions  with  the  proposal  Offerors. 

•  TARDEC  RAM  Engineers  provide  SSEB  result  briefings  to 
the  Source  Selection  Authority  (SSA). 
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RAM  Support  at  Design  Reviews 


•  TARDEC  RAM  Engineers  support  the  following 
major  acquisition  program  reviews: 

•  Start  of  Work  Meeting  (SOWM) 

•  System  Requirements  Review  (SRR) 

•  Preliminary  Design  Review  (PDR) 

•  Critical  Design  Review  (CDR) 

•  Test  Readiness  Review  (TRR) 

•  Program  Management  Review  (PMR) 

•  Responsibilities  of  RAM  Engineers  at  program 
reviews  include: 

•  Ensuring  Vendor  correctly  interprets  Purchase 
Description  (PD)  RAM  requirements 

•  Reviewing  Vendor  RAM  predictions  and  analyses 

•  Verifying  that  the  Vendor  is  adequately  prepared  to 
enter  formal  government  testing,  and  tracking  test 
progress  throughout  the  test  phase 
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RDECOM 


Benefits  of  RAM  Engineer  Participation 
at  Program  Design  Reviews 


•  Benefits  of  having  RAM  representation  at  formal 
reviews: 

•  SOWM/SRR:  Opportunity  to  meet  the  Vendors’  RAM 
sub-integrated  Product  Team  (sub-IPT)  and  establish 
strong  communication.  RAM  engineers  verify  that 
the  RAM  requirements  are  interpreted  correctly. 

•  PDR/CDR:  RAM  Subject  Matter  Experts  (SME)  can 
review  RAM  predictions/  analyses  and  ask  questions 
directly  to  the  Vendor’s  RAM  sub-IPT. 

•  TRR/PMR:  RAM  engineers,  Test  and  Evaluation  IPT 
Chiefs,  and  Test  Managers  assess  the  Vendor’s 
readiness  for  formal  testing.  Once  testing  has 
commenced,  RAM  Engineers  can  accurately  report 
on  RAM  test  status. 


TECHNOLOGY  DRIVEN.  WARFIGHTER  FOCUSED.  J 


RDECOM/  Performance  and  Requirement 

Evaluation  during  RAM  test 


•  R/M  Test  Planning  -  Test  Site/Sample  size 
determination 

•  Pre-test  meeting/test  readiness  meeting 

•  Test  Data  Collection/  Test  Incident  Report 

•  Army  Test  Incident  Report  (ATIR)  Database  and 
COGNOS 

•  Failure  Analysis  and  Corrective  Action  Report 

•  Continuous  monitor  of  TIRs  and  FACAR 

•  Pre-Scoring  Conference  and  TIR  review 
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RDECOMjk  Performance  and  Requirement 

Evaluation  during  RAM  test 


•  Scoring  Conference 

•  Failure  Analysis  and  Corrective  Action 
Implementation 

•  Continue  Testing  with  Design 
Changes/Corrective  Action 

•  RAM  Assessment  Conference 

•  Final  RAM  performance  Evaluation 

•  RAM  requirement  calculation/verification 
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Scoring  IPT 


Z-^Z. _ \\ _ 1 


•  Purpose:  To  review  all  TIRs  generated  from  RAM  test  and  establish 
common  data  base  for  evaluating  the  R/M  performance  of  the 
system 

•  Guiding  Document:  FDSC  (published  by  C  D  with  materiel 
developer’s  input) 

•  Scoring  IPT  is  made  up  of  three  voting  members  representing 

•  Materiel  Developer  (PM) 

•  Combat  Developer  (School) 

•  Independent  Evaluator  (AEC) 

•  The  dissenting  voter  can  submit  minority  opinion 

•  it  gets  attached  to  the  minutes. 

•  Development  Test  Scoring  IPT 

•  Materiel  Developer  chairs  the  scoring  IPT 

•  Operational  Test  Scoring  IPT 

•  Independent  Evaluator  chairs  the  scoring  IPT 

•  Contractors  can  present  the  results  of  failure  analysis  that  may 
assist  voting  members 

•  The  scoring  is  government  unilateral  decision 

•  The  minutes  that  has  the  official  scores  are  published  by  the 
chairing  agency. 
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rdecom^  Preparation  for  Scoring  IPT 


•  During  test  phase  scoring  IPT  may  be  held 
as  often  as  few  weeks  depending  on  the 
program  and  number  of  TIRs 

•  Pre-scoring  conference  with  contractor 

•  Review  of  TIRs  with  contractor  to  fully  take  their  take 
on  each  failures 

•  Materiel  Developer  carries  the  burden  of  proof  and 
need  the  maximum  preparation  for  the  scoring  process 


•  On-line  Scoring  tool  available  (real  time 
scoring) 

•  Face-to-face  meeting  only  for  those  incidents  that  are 
not  agreed  upon 

•  Cuts  almost  ~60%  of  meeting  time 
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Reports  (TIRs) 


•  Development  Test 

•  ATEC  writes  and  publishes  TIR  for  all  incidents  during 
development  test 

•  Testers  run  the  test  and  the  result  of  the  test  is  technical 
evaluation 

•  The  result  is  evaluated  against  SPEC  requirement 


•  Operational  Test 

•  TIRs  will  be  written  by  test  director  (depends  on  OTC) 

•  TIRs  needs  to  be  released  by  Dag  (Data 
Authentication  Group) 

•  Independent  Evaluator  chairs  the  scoring  IPT 

•  No  supplemental  information  is  available  in  addition  to  the 

TIRs 

•  Scoring  process  is  similar  to  DT  scoring 

•  The  result  is  evaluated  against  the  User’s  requirement 
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rdecom* ^  Reliability  Data  Analysis 


•  Reliability  Analysis  must  be  discussed 
up  front  and  consensus  should  be 
reached  on: 

•  FDSC  -  Failure  Definition  Scoring  Criteria 

•  Failure  Categories 

•  Inherent  vs.  Induced  Reliability 

•  Mission  Profile  and  Life  Variable 

•  Data  Grouping  and  Modeling 

•  Instantaneous  vs.  Cumulative  Reliability 
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RDECOM 


FDSC  -  Failure  Definition 
Scoring  Criteria 


•  FDSC  is  Contractual  Document  that  defines 

•  Failure/non-Failure  Event 

•  Test  related  Event 

•  Severity  of  Failure  as  it  relates  to  the  Mission 

•  Cause  of  the  Failure 

•  FDSC  is  prepared  as  required  by  Army  Regulation 
70-1 ,  Army  Acquisition  Policy. 

•  FDSC  is  being  used  through  out  the  test  for 
Scoring  purposes,  hence  it  is  a  major  document 
for  RAM  Assessment 
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B^u^Pajlure  |y|0(je  Qateg0rjes  ^4^? 


•  Performance  FM  -  FM  is  repeatable  with  100% 
probability  of  failure  for  the  given  procedure/conditions. 
(Example:  TDS  overheating) 


•  Software  FM  -  same  as  above,  but  software  related. 


•  Quality  FM  -  happens  when  vehicle  is  not 
built/maintained/operated  as  designed  and  is  not 
repeatable  after  fixing  (probability  of  failure  =0%).  Can 
be  broken  down  into  mifial  Quality,  Maintenance, 
Operator  error,  etc.  (Example:  Improperly  installed 
harness,  turret  lock  bended,  etc.) 


•  Potential  Reliability  FM  -  happens  when  vehicle  was 
built/maintained/operated  as  designed/intended; 
probability  of  failure  is  greater  than  0%  and  less  than 
100%:  usually  happens  due  to  wear  out,  environment, 
insufficient  design,  manufacturing  variability,  etc. 
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**  Failure  Mode  Management 


raining  and  Manuals 
Design  Simplifications 
Management  of  Maintenance  Actions 


□  Human 

□  Performance 
■  Quality 

□  Reliability 


Robust  Design 
Adequate  Design  Margin^ 
DFMEA 

Step-wise  Verificatioi 
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Reliability  Assessment 


*  A  process  of  recognizing  successful  Corrective 
Action  (CA)  per  failure  mode 

•  Scoring  members  will  evaluate  the  CA  to  assess 
out  certain  failures  based  on  the  proven  CA  during 
test 


*  Fully  proven  successful  CA  :  70-80  %  assessed 
out  rate 

•  Assessed  Reliability  Value  will  be  published  in 
the  final 

test  report  and  it  is  the  official  Reliability  value 
for  the  test 
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rdfkom^  Reliability  Growth 


•  What  is  a  Reliability  Growth? 

•  Why  Reliability  Growth  program? 

•  Reliability  Growth  during  Concept  phase 

•  Reliability  Growth  during  Design  phase 

•  Reliability  Growth  during  Test  phase 
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What  is  a  Reliability 


•  Reliability  Growth  is  the  increase  in  the  true  reliability  of  a  system  as 
a  result  of  failure  mode  discovery,  analysis  and  effective  correction 

•  Reliability  Growth  Management  is  the  systematic  planning  for 
reliability  achievement  by  controlling  the  ongoing  rate  of 
achievement  by  the  allocation  and  reallocation  of  program  resources 
based  on  comparisons  between  planned  and  demonstrated  reliability 
values. 

•  Reliability  Growth  Planning  addresses  program  schedules,  amount  of  testing, 
resources  available,  and  the  realism  of  the  test  program  in  achieving  its 
requirements.  Reliability  growth  planning  is  portrayed  and  quantified  through  a 
reliability  growth  planning  curve. 

•  Reliability  Growth  Tracking  is  an  area  of  reliability  growth  that  provides 
management  the  opportunity  to  gauge  the  progress  of  the  development  effort  by 
quantifying  the  demonstrated  reliability  of  a  system  throughout  its  test  program. 

•  Reliability  Growth  Projection  is  an  area  of  reliability  growth  that  focuses  on 
quantifying  the  reliability  that  could  be  achieved  if  observed  failure  modes 
inherent  to  the  system  are  mitigated  by  a  specified  level  of  fix  effectiveness. 
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RDECOM 


Cost  of  Achieving  Reliability 


Only  ONE  way  to  improve  Reliability: 

Identify  the  failure  mode,  its  cause,  and  remove  or  mitigate  it 


Only  THREE  periods  in  time  to  do  it: 


$  Cost 


1.  After  hardware  fails  in  the  field 

2.  After  hardware  is  built  but 
before  its  fielded 

3.  Before  hardware  is  built 


Where  should  $  and  effort  be  placed? 

Q  2  , 


Time  Period 


H  3.  Design  Phase 
□  2.  Before  Field 
■  1.  After  Fielded 
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Reliability  Growth  during 
Design  Phase 


•  Reliability  Growth  management  during  Design 
Phase 

•  Design  for  Reliability  (DFR)  is  the  tool  for  Reliability  Growth 
during  design  phase 

•  DFR  activities  (Reliability  Program  Plan  -  CDRL) 

•  R  Prediction 

•  R  Allocation 

•  FMEA 

Fault  Tree  Analysis 

•  DART  Process 

•  The  growth  resulting  from  DFR  activities  tracked  on  the 
contractor’s  planning  growth  curve  (Reliability  Case  Report 
-  CDRL) 

•  Results  in  design  and  redesign  to  incorporate  corrective 
action 

•  Updating  R  prediction  and  allocation 
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Must  Raise  Assessment 
Closer  Threshold 


/  //  \\  V 


Increase  Design  Effectiveness  Using 
DFR  Methodology 


Manage  Growth  Potential 


Allocated  Target 


DFR  Enables 
Higher  Initial  MMBSA 
At  Start  Of  Test 


R  Prediction 
R  Allocation  \ 
FMEA 
Fault  Tree 
Analysis 


Increase 

Initial 

MMBSA 


v.  _ 

Current  Assessment 

Design  Phase  ■ 


4 


PQT 
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•  Reliability  Growth  management  during  Test  phase 
(PQT) 

•  Reliability  growth  through  Test,  Analyze,  Fix  and  Test 
(TAFT) 

•  Test  sample  size  and  test  length  needs  to  be  sufficient 
to  grow  R 

•  Failure  Analysis  Corrective  Action  Reporting  (FACAR) 
system  is  a  tool  for  Reliability  Growth  during  test  phase 

•  Tracking  R  using  Growth  Model  such  as  AMSAA/Crow 
model 

•  Provide  management  an  opportunity  to  gauge  the 
progress  of  the  development  effort  by  quantifying  the 
demonstrated  reliability  of  a  system  throughout  its  test 

•  If  data  or  test  length  does  not  support  any  growth 
model,  the  assessment  conference  will  convene  and 
assess  the  fix  effectiveness  of  the  corrective  actions 
that  were  implemented 
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RDECOM 


Test-in  Reliability 


Test 
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